Can We Automate Scientific Reviewing?
نویسندگان
چکیده
The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications. At the same time, review each paper is a laborious process that must be carried out subject matter experts. Thus, providing high-quality reviews this growing number papers significant challenge. In work, we ask question “can automate reviewing? ”, discussing possibility using natural language processing (NLP) models to generate peer for papers. Because it non-trivial define what “good” first place, discuss possible evaluation metrics could used judge success task. We then focus on machine learning domain collect dataset domain, annotate them with different aspects content covered review, train targeted summarization take as input output. Comprehensive experimental results test set show while system-generated are comprehensive, touching upon more than human-written reviews, generated texts less constructive factual all except explanation core ideas papers, which largely factually correct. Given these results, pose eight challenges pursuit good generation system together potential solutions, which, hopefully, will inspire future research direction. make relevant resource publicly available use research: https://github. com/neulab/ReviewAdvisor. addition, our conclusion not yet ready high-stakes settings provide demo, ReviewAdvisor (http://review.nlpedia.ai/), showing current capabilities failings state-of-the-art NLP at task (see demo screenshot A.2). A written proposed can found A.1.
منابع مشابه
Can We Automate Compliance to Collision Avoidance Resolution Advisories?
The Traffic alert and Collision Avoidance System (TCAS) was designed to prevent mid-air collisions by portraying a time-critical “resolution advisory” (RA) to the pilot. Complying with a TCAS RA is generally considered the safest course of action and takes priority over all other forms of air traffic management; however, pilot compliance with TCAS RAs is surprisingly low. One proposed solution ...
متن کاملReviewing scientific papers.
Guidelines for the preparation of structured reports for laboratory studies, clinical studies, epidemiological studies, and observations could be useful. A structured report should prompt reviewers to acknowledge familiarity or otherwise with method, background papers, and so on. New reviewers could draw up their own system. Reports should be helpful and courteous so that whether or not the pap...
متن کاملReviewing, Reviewers and the Scientific Enterprise
Despite their critical importance to the scientific enterprise, reviewers receive no formal training and reviewing has become a skill that they pick up through trial and error. Additionally, because most reviewers do not receive any feedback on their performance, any bad reviewing habits become entrenched over time. This has contributed to significant and unnecessary anxiety about reviewing and...
متن کاملCan We Add Auricular Composite Graft to Our Rhinoplasty Armamentarium?
BACKGROUND The ala of the nose, with its particular texture and characteristics, poses both aesthetically and functionally intriguing challenges and is rather problematic regarding choices for reconstructive methods. Both flaps and grafts have been used to restore natural structure of nasal ala. The present study summarizes a ten-year experience of reconstructive surgery using small composite...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Artificial Intelligence Research
سال: 2022
ISSN: ['1076-9757', '1943-5037']
DOI: https://doi.org/10.1613/jair.1.12862